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Central  to  the  federal  response  to  the 
2009  MINI  influenza  pandemic  was  a 
vaccination  program  unprecedented 
in  its  size  and  scope  in  the  United 
States.  The  2009  MINI  monovalent  in¬ 
fluenza  vaccines  were  approved  by  the 
US  Food  and  Drug  Administration 
(FDA)  of  the  US  Department  of  Health 
and  Human  Services  (HHS)  as  a  strain 
change  to  each  manufacturer’s  sea¬ 
sonal  influenza  vaccine.  There  is  con¬ 
siderable  experience  with  seasonal  in¬ 
fluenza  vaccine  development  and 
production.  Influenza  vaccines  have  a 
long  track  record  of  safety  and  effec¬ 
tiveness  in  the  United  States.  The  2009 
H1N1  monovalent  influenza  vaccines 
underwent  the  same  testing  and  lot- 
release  procedures  that  are  in  place 
for  seasonal  influenza  vaccines.  Con¬ 
sequently,  the  safety  profiles  of  2009 
H1N1  monovalent  influenza  vaccines 
were  anticipated  to  be  similar  to  the 
excellent  safety  profile  of  seasonal  in¬ 
fluenza  vaccines.  In  addition,  the  safety 
of  the  2009  HI  N1  monovalent  influenza 
vaccines  was  carefully  assessed  in 
multiple  clinical  trials.  The  sample 
sizes  of  these  trials  limited  the  ability 
to  detect  rare  adverse  events.  Popula¬ 
tions  at  high  risk,  such  as  those  with 
chronic  diseases,  are  sometimes  not 
well  represented  in  clinical  studies; 
however,  additional  efforts  were  made 
for  2009  H1N1  monovalent  influenza 
vaccines  to  include  in  clinical  trials 
pregnant  women  and  people  with  un¬ 
derlying  illness  such  as  asthma  and 
HIV  infection. 

A  key  component  of  any  immunization 
program  is  postlicensure  safety  moni¬ 
toring.'  Such  a  monitoring  system 
must  have  the  abilityto  quickly  identify 
and  characterize  adverse  events  after 
vaccination.  A  vaccine-safety  monitor¬ 
ing  system  should  have  the  capacity  to 
distinguish  a  potential  increased  risk 
of  an  adverse  event  caused  by  the  vac¬ 
cine  from  events  that  occur  as  part  of 
the  background  incidence  of  these  dis¬ 


eases  in  temporal  association  with 
vaccination.'  Broad  and  integrated  ef¬ 
forts  that  monitor  vaccine  safety  after 
licensure  are  important  for  rapidly 
and  effectively  defining  the  safety  pro¬ 
file  of  a  vaccine. 

The  vaccine-safety  monitoring  process 
includes  3  primary  activities: 

1.  Signal  detection,  strengthening, 
and  verification  involve  detection  of 
medical  events  after  vaccination 
and  an  evaluation  of  whether  these 
adverse  events  could  be  occurring 
more  frequently  after  vaccination 
than  expected  by  chance  alone.  For 
this  activity,  potential  signals  are 
evaluated  to  assess  whether  they 
warrant  further  investigation.  This 
evaluation  includes  examining  ifthe 
reported  events  were  well  defined 
and  properly  coded,  if  the  events 
were  reported  from  multiple  re¬ 
porting  sources  or  only  by  a  few,  or 
ifthere  was  a  pattern  of  association 
between  vaccination  and  adverse 
events  (eg,  temporal  or  demo¬ 
graphic  relationships  or  subpopu¬ 
lations  affected).  Efforts  are  also 
made  to  look  for  unexpected  clini¬ 
cal  clusters  and  positive  rechal¬ 
lenges  (symptoms  that  reoccur  af¬ 
ter  readministration  of  vaccine). 

2.  Assessment  of  association  involves 
evaluating  whether  there  is  an  as¬ 
sociation  between  vaccination  and 
an  adverse  event.  If  an  association 
between  the  vaccine  and  the  out¬ 
come  is  found,  it  is  important  to  de¬ 
termine  the  magnitude  of  the  asso¬ 
ciation  and  whether  potential 
subpopulations  are  at  increased 
risk. 

3.  Assessment  of  the  evidence  and 
causality  involves  evaluating 
whether  the  available  science  fa¬ 
vors  acceptance  or  rejection  of  a 
relationship  between  the  vaccine 
and  the  adverse  event,  which  often 
requires  population-based  active 
surveillance  and  formal  epidemio¬ 


logic  studies.  Causality  assess¬ 
ments  typically  include  consider¬ 
ation  of  the  strength  or  magnitude 
of  the  association,  consistency, 
specificity,  temporality,  biological 
gradient,  biological  mechanism,  co¬ 
herence,  experimental  evidence, 
and  analogy.2 

Before  the  initiation  of  the  2009  H1N1 
monovalent  influenza  vaccination  pro¬ 
gram,  a  number  of  existing  systems 
addressed  these  3  activities  of  the 
safety  monitoring  system.  The  HHS  led 
an  effort  to  enhance  existing  systems 
and  integrate  new  vaccine-safety  mon¬ 
itoring  systems.  These  efforts  were 
deemed  to  be  integral  to  the  immuni¬ 
zation  program  given  its  size  and 
prominence  and  residual  concerns 
about  the  1976  pandemic  influenza 
vaccination  program  in  which  the  vac¬ 
cine  was  associated  with  Guillain- 
Barre  syndrome  (GBS).  In  this  article, 
we  discuss  the  existing  systems  for 
monitoring  vaccine  safety  and  en¬ 
hancements  that  were  made  to  sup¬ 
port  the  2009  H1N1  influenza  vaccina¬ 
tion  program. 

FEDERAL  SYSTEMS  FOR 
MONITORING  VACCINE  SAFETY 
BEFORE  THE  2009  MINI 
MONOVALENT  INFLUENZA  VACCINES 

The  US  vaccine-safety  system  is  com¬ 
posed  of  a  number  of  programs  man¬ 
aged  by  federal  agencies  within  the 
HHS,  the  Department  of  Veterans  Af¬ 
fairs  (VA),  and  the  Department  of  De¬ 
fense  (DoD).  A  brief  description  of  the 
vaccine-safety  system  before  the  2009 
H1N1  monovalent  influenza  vaccines 
follows.  A  more  comprehensive  review 
of  federal  vaccine-safety  monitoring 
systems  is  available  elsewhere.® 

Signal  Detection,  Strengthening, 
and  Verification 

The  Vaccine  Adverse  Event  Reporting 
System  (VAERS),  established  in  1990,  is 
co-managed  by  the  FDA  and  the  Cen- 
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tens  for  Disease  Control  and  Preven¬ 
tion  (CDC).  The  VAERS  is  a  national  pas¬ 
sive  surveillance  system  that  receives 
reports  of  potential  adverse  events 
from  many  sources  including  health 
care  providers  and  the  public.  The 
VAERS  can  assess  early  indicators  of  a 
possible  vaccine-safety  problem  that 
may  present  as  new  or  unusual  ad¬ 
verse  events  or  patterns  of  reports. 
For  example,  in  1999,  the  VAERS  was 
the  first  postlicensure  source  to  signal 
an  increased  risk  of  intussusception 
after  the  first  dose  of  the  rotavirus  vac¬ 
cine  RotaShield  (Wyeth  Laboratories, 
Marietta,  PA).  This  signal  was  later 
confirmed  to  be  a  true  association.'^ 
Because  the  VAERS  is  a  passive  report¬ 
ing  system  for  those  who  have  been 
vaccinated,  it  is  not  able  to  determine 
how  many  people  have  been  vacci¬ 
nated  or  the  rates  of  events  among 
persons  not  vaccinated.  In  addition,  as 
a  passive  reporting  system,  the  VAERS 
suffers  from  underreporting  and  in¬ 
complete  reporting.  Consequently,  it  is 
useful  for  signal  detection,  whereas 
other  systems  are  used  to  determine 
associations  between  vaccination  and 
adverse  events.® 

Four  related  systems  take  advantage 
of  administrative  and  clinical  data 
available  in  health  care  systems.  The 
Vaccine  Safety  Datalink  (VSD),  admin¬ 
istered  by  the  CDC,  uses  rapid  cycle 
analysis  (RCA)  to  strengthen  and  verify 
signals  of  prespecified  outcomes  and 
to  assess  associations  (discussed 
more  later).  The  VSD  consists  of  a 
large  linked  database  of  8  managed 
care  organizations  (MCOs)  that  cover 
~9  million  people,  or  3%  of  the  US  pop¬ 
ulation.  MCOs  contribute  demographic 
and  vaccination  data  linked  to  diag¬ 
noses  from  medical  encounters.  The 
VSD  can  be  used  for  a  broad  range  of 
studies,  because  it  has  fairly  complete 
data  on  vaccine  exposures  and  health 
outcomes  coupled  with  chart  review 
as  needed. 


The  DoD  uses  the  Defense  Medical  Sur¬ 
veillance  System  (DMSS)  as  a  central  re¬ 
pository  of  medical  surveillance  data  for 
the  US  armed  forces  (~1  million  per¬ 
sons).  Military  health  records  in  this  sys¬ 
tem  can  be  used  to  examine  medical  out¬ 
comes  after  vaccination.®-®  The  DMSS 
can  be  used  for  signal  detection 
through  data-mining  as  well  as  signal 
strengthening  and  verification. 

The  VA  has  collaborated  with  the  FDA 
since  2008  to  use  data  from  the  VA  na¬ 
tional  databases  to  detect  potential 
vaccine-safety  signals  among  the  vet¬ 
eran  and  VA  employee  populations. 
Early  evaluations  have  assessed  the 
safety  of  influenza  and  pneumococcal 
vaccines. 

The  FDA  and  the  Centers  for  Medicare 
and  Medicaid  Services  initiated  a  pilot 
project  in  2006  to  assess  the  feasibility 
of  using  Medicare  data  for  prospective 
rapid  safety  assessment  of  vaccines 
administered  in  the  Medicare  popula¬ 
tion.  Medicare  insures  persons  who 
are  65  years  old  or  older  and  younger 
persons  with  disabilities  or  end-stage 
renal  disease.  More  than  45  million 
persons  (including  38  million  people 
aged  >65  years)  are  enrolled  in  Medi¬ 
care.  Claims  data  are  available  for 
—35  million  persons  with  fee-for- 
service  Medicare.  Although  largely  lim¬ 
ited  to  older  populations,  the  size  of 
the  enrolled  population  provides  op¬ 
portunities  to  evaluate  rare  adverse 
events  that  other  systems  may  not  be 
able  to  address. 

Assessment  of  Association 

The  VSD  is  the  primary  system  for  as¬ 
sessing  associations  between  vaccines 
and  adverse  events,  because  it  links  vac¬ 
cination  status  and  health  outcomes, 
which  provides  the  infrastructure  to  rap¬ 
idly  test  hypotheses.  The  VSD  is  used  for 
a  large  number  of  vaccine-safety  studies 
and  is  widely  considered  to  be  the  back¬ 
bone  of  the  US  vaccine-safety  system.  A 
broad  range  of  study  designs  are  used 


by  the  VSD,  including  cohort,  case- 
control,  and  self-controlled  case- 
series  studies. 

The  Defense  Medical  Surveillance  Sys¬ 
tem  has  capabilities  similarto those  of 
the  VSD  and  is  widely  used  within  the 
DoD  to  investigate  a  broad  array  of  ex¬ 
posures  and  outcomes  that  are  often 
unique  to  the  military  population. 

Additional  studies  may  be  conducted, 
in  coordination  with  state  health  de¬ 
partments  and/or  epidemic  intelli¬ 
gence  service  officers,  as  part  of 
outbreak  investigations  to  evaluate 
potential  vaccine-safety  concerns.  Be¬ 
cause  these  studies  typically  investi¬ 
gate  rare  events,  case-control  studies 
can  be  conducted,  such  as  those  exam¬ 
ining  intussusception  after  the  rotavi¬ 
rus  vaccine.  A  variety  of  approaches 
can  be  used  to  identify  cases  for  case- 
control  studies,  including  active  sur¬ 
veillance  systems.'*'® 

Assessment  of  the  Evidence  and 
Causality 

In  2001,  the  CDC  established  the  Clinical 
Immunization  Safety  Assessment  (CISA) 
Network.  Centers  in  the  CISA  Network  in¬ 
vestigate  the  pathophysiologic  mecha¬ 
nisms  and  biological  risks  of  vaccine 
adverse  events,  which  are  important 
considerations  for  causality  assess¬ 
ment.  These  centers  conduct  in-depth 
immunologic,  pathologic,  and  genetic 
assessments  to  elucidate  underlying 
mechanisms  of  vaccine  adverse  events. 
In  addition  to  contributing  to  the  body  of 
evidence  needed  for  causality  assess¬ 
ments,  the  CISA  Network  assists  clini¬ 
cians  in  evaluating  and  managing  the 
conditions  of  people  with  possible  vac¬ 
cine  adverse  reactions. 

For  independent  expert  review  of  the 
evidence  for  causality  of  particular 
vaccines  and  adverse  events,  the  US 
government  has  periodically  relied  on 
independent,  nongovernmental  review 
through  the  Institute  of  Medicine  (lOM) 
of  the  National  Academies.  The  lOM 
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conducted  its  first  report  on  vaccine 
safety  in  1977  and  has  subsequently 
published  6  adverse-event  reviews. 
These  reviews  cover  a  range  of  ad¬ 
verse  events  such  as  encephalopathy, 
GBS,  and  sudden  infant  death  syn¬ 
drome.  Reviewers  consider  potential 
biological  mechanisms,  epidemiologic 
and  clinical  data,  the  burden  of  the  ad¬ 
verse  event,  the  burden  of  the  vaccine- 
preventable  disease,  and  salience  to 
the  public.  The  lOM  is  currently  con¬ 
ducting  a  comprehensive  review  of  the 
epidemiologic,  clinical,  and  biological 
evidence  regarding  adverse  health 
events  associated  with  specific  vac¬ 
cines  covered  by  the  National  Vaccine 
Injury  Compensation  Program. 

ENHANCING  THE  VACCINE-SAFETY 
MONITORING  SYSTEM  TO  SUPPORT 
THE  2009  H1N1  MONOVALENT 
INFLUENZA  VACCINE  PROGRAM 

The  2009  EI1N1  monovalent  influenza 
vaccine  program  prompted  the  federal 
government  and  the  National  Vaccine 
Advisory  Committee  (NVAC),  a  federal 
advisory  committee  that  provides  rec¬ 
ommendations  to  the  Assistant  Secre¬ 
tary  for  Health,  to  assess  the  capacity 
of  the  existing  safety-monitoring  sys¬ 
tems.  The  NVAC  made  5  recommenda- 
tions'“'"  regarding 2009  MINI  vaccine- 
safety  monitoring:  (1)  enhance  active 
surveillance  for  signal  confirmation 
and  evaluation  of  possible  associa¬ 
tions  between  vaccines  and  adverse 
events;  (2)  establish  a  transparent  and 
independent  review  of  vaccine-safety 
data  as  it  accumulates;  (3)  develop  and 
disseminate  a  federal  plan  to  monitor 
2009  MINI  monovalent  influenza  vac¬ 
cine  safety;  (4)  assemble  background 
rates  of  adverse  events  that  occur  in 
the  general  population;  and  (5)  de¬ 
velop  and,  when  possible,  test  in 
advance  a  strong  and  organized 
response  to  scientific  and  public  con¬ 
cerns  about  vaccine  safety.  Actions 
taken  to  respond  to  these  recommen¬ 
dations  are  described  below. 


Signal  Detection,  Strengthening, 
and  Verification 

A  number  of  efforts  were  made  to  fa¬ 
cilitate  adverse-event  reporting.  The 
GDC  developed  an  influenza  vaccina¬ 
tion  record  card  for  immunization  pro¬ 
viders  to  give  to  the  vaccine  recipient 
with  the  vaccine.  The  card  included  in¬ 
formation  on  how  to  report  an  adverse 
event  to  the  VAERS.  Providers  were 
asked  to  record  vaccine  type,  dose, 
date,  and  lot  number  on  tbe  card.  At 
the  time  of  vaccination,  cards  were 
given  to  the  vaccine  recipient  (or  care¬ 
giver)  to  keep  for  1  year  after  the  last 
2009  MINI  influenza  vaccine  received. 
All  reports  of  serious  events  to  the 
VAERS  were  reviewed  daily,  and  the 
frequency  of  events  after  2009  MINI 
monovalent  influenza  vaccines  was 
compared  with  the  frequency  of  events 
after  seasonal  influenza  vaccines. 
Medical  records  from  people  who  ex¬ 
perienced  serious  adverse  events 
were  quickly  obtained  and  reviewed.  In 
addition,  the  GDC  actively  monitored 
newspaper  articles  and  blogs  to 
quickly  identify  public  concerns  that 
might  indicate  a  vaccine-safety  signal. 

The  HHS  Indian  Health  Service  devel¬ 
oped  and  deployed  in  May  2009  the  In¬ 
fluenza  Awareness  System.  This  sys¬ 
tem  covered  seasonal  and  2009  H1N1 
vaccination  and  potential  vaccine  ad¬ 
verse  events. 

The  Real  Time  Immunization  Monitor¬ 
ing  System  (RTIMS),  developed  at 
Johns  Hopkins  University  and  spon¬ 
sored  by  the  GDG,  used  an  automated 
Web-based  active  surveillance  system 
to  track  adverse  events  among  vac¬ 
cines.  Previously,  the  DoD  piloted  a 
similar  electronic  monitoring  system 
to  assess  patient  experiences  after 
smallpox  and  seasonal  influenza  vacci¬ 
nation. jhe  RTIMS  was  pilot-tested 
by  Johns  Hopkins  in  the  2008-2009  in¬ 
fluenza  season.  During  the  2009-2010 
influenza  season,  vaccine  recipients  ei¬ 
ther  gave  permission  to  be  contacted 


for  follow-up  at  the  time  of  their  immu¬ 
nization  or  chose  to  log  onto  a  Web  site 
and  answer  an  electronic  question¬ 
naire.  Follow-up  e-mail  reminders  pro¬ 
vided  a  link  to  questionnaires  to 
provide  answers  to  a  series  of 
bealth-related  questions  at  various 
time  points  after  vaccination.  An¬ 
swers  were  analyzed  by  using  a  rule- 
based  algorithm  in  realtime.  Like  the 
VAERS,  this  system  only  collected 
data  on  vaccine  recipients  and 
lacked  data  from  a  nonvaccinated 
comparison  group.  Gomparisons  be¬ 
tween  types  of  vaccines  (eg,  sea¬ 
sonal  versus  2009  H1N1  monovalent 
influenza  vaccines  and  live  versus  in¬ 
activated)  were  made.  RTIMS  investi¬ 
gators  collected  supplemental  infor¬ 
mation  by  follow-up  telephone  and 
e-mail  contact  with  vaccine  recipi¬ 
ents  and  their  health  care  providers. 
Reporting  rates  for  adverse  events 
among  those  who  received  2009 
H1N1  monovalent  influenza  and  sea¬ 
sonal  influenza  vaccines  were  as¬ 
sessed.  Strengths  and  limitations  of 
the  VAERS  and  RTIMS  are  summa¬ 
rized  in  Table  1. 

A  prioritized  list  of  potential  adverse 
events,  along  with  plausible  time  win¬ 
dows  for  tbeir  occurrence  to  be  re¬ 
lated  to  the  vaccine,  was  developed  on 
the  basis  of  epidemiologic  associa¬ 
tions  with  current  or  past  vaccines  or 
on  biological  plausibility  (see  Appen¬ 
dix).  These  prespecified  outcomes 
were  investigated  by  using  rapid  sur¬ 
veillance  methodologies  (RGA)  in  the 
VSD,  Defense  Medical  Surveillance  Sys¬ 
tem,  and  other  systems  described  be¬ 
low.  For  example,  background  rates 
were  calculated  for  GBS  and  other  po¬ 
tential  neurologic  illnesses  of  the  cen¬ 
tral  nervous  system  (such  as  acute 
disseminated  encephalomyelitis,  en¬ 
cephalitis,  myelitis,  and  optic  neuritis) 
for  comparison  to  rates  after  vaccina¬ 
tion. Additional  health  outcomes 
could  have  been  added  for  RGA  had  po- 
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TABLE  1  Signal  Detection  Programs  Strengths  and  Limitations 


Data 

Source 

Strengths 

Limitations 

VAERS 

Nationwide 

Passive  system  prone  to  underreporting, 
lack  of  consistency  in  reporting,  and 

Near  real-time 

variable  quality  of  reported  information 

Can  detect  rare  or  unexpected  events 

Not  well-defined  denominators  only  available 
from  other  sources 

Lot-specific  surveillance 

Data-mining 

No  comparison  group 

RTIMS 

Actively  soliciting  symptoms  after  vaccination 

Recruiting  large  numbers  of  persons  is 

by  enrolling  participants  the  same  day  or 

challenging 

shortly  after  receipt  of  vaccine  (14  000 

2009  H1N1  monovalent  influenza  vaccine 
doses  captured) 

Sample  size  inadequate  for  rare  events 

Can  target  subpopulations 

Surveys  completed  after  vaccination 

tential  signals  arisen  during  the 
course  of  the  vaccine  program,  which 
to  date  has  not  been  the  case.  RCA 
compares  the  rates  of  prespecified 
events  that  occurred  after  2009  H1N1 
monovalent  influenza  vaccination  with 
the  number  of  events  that  would  be  ex¬ 
pected.  Expected  rates  were  initially 
calculated  on  the  basis  of  persons  who 
received  seasonal  influenza  vaccine  in 
previous  years,  which  allowed  com¬ 
parisons  to  be  made  on  an  ongoing 
and  regular  basis.  Final  (end-of- 
season)  analyses  will  calculate  ex¬ 
pected  rates  among  persons  who  re¬ 
ceived  the  2009  MINI  monovalent 
influenza  vaccine  during  different  time 
periods  (self-controlled  analysis).  The 
self-controlled  analysis  has  advan¬ 
tages  over  making  comparisons  to  his¬ 
toric  controls  (those  who  received  sea¬ 
sonal  influenza  vaccines  in  previous 
years),  because  it  does  not  suffer  from 
potential  bias  that  may  be  caused  by 
differences  in  the  underlying  popula¬ 
tions  (those  who  receive  seasonal  in¬ 
fluenza  vaccines  may  be  different  from 
those  who  receive  2009  MINI  vac¬ 
cines).  However,  self-controlled  analy¬ 
sis  is  less  timely,  because  it  requires 
additional  time  windows  used  for  com¬ 
parison  purposes  to  elapse.  For  both 
of  these  analyses,  power  is  primarily 
driven  by  the  number  of  persons  who 
receive  2009  H1N1  monovalent  influ¬ 


enza  vaccines  captured  in  the  surveil¬ 
lance  systems. 

The  VSD,  DoD,  VA,  and  Centers  for  Medi¬ 
care  and  Medicaid  Services  systems 
can  use  data  on  who  was  vaccinated 
and  assess  their  health  outcomes.  Al¬ 
though  the  primary  purpose  of  these 
systems  during  2009  MINI  vaccine- 
safety  monitoring  was  signal  detec¬ 
tion,  strengthening,  and  verification, 
they  have  the  ability  to  assess  associ¬ 
ations  if  needed. 

Assessment  of  Association 

Several  systems  were  used  to  assess 
an  association  in  addition  to  the  VSD, 
which  historically  met  this  need.  The 
VSD  has  been  a  critical  element  of  the 
vaccine-safety  system,  especially  in 
assessing  associations  between  vac¬ 
cines  and  adverse  events.  However, 
the  NVAC  noted  that  the  capacity  of  the 
VSD  to  meet  the  needs  for  monitoring 
safety  during  the  2009  H1N1  monova¬ 
lent  influenza  vaccine  program  may  be 
limited  for  2  primary  reasons.  First, 
despite  its  large  size,  the  ability  of  the 
VSD  to  rapidly  assess  emerging  issues 
might  not  be  timely  enough,  particu¬ 
larly  for  extremely  rare  adverse  events 
and  events  that  occur  among  subpopu¬ 
lations.  Second,  MCOs  that  participate 
in  the  VSD  may  not  necessarily  capture 
significant  portions  of  2009  HI  N1  mon¬ 
ovalent  influenza  vaccine  adminis¬ 


tered  at  sites  outside  ofthe  MCOs  such 
as  health  departments,  schools,  and 
mass-vaccination  clinics.  To  address 
these  limitations,  several  approaches 
(described  below)  were  used  to  en¬ 
hance  assessment  of  associations. 

The  Post-licensure  Rapid  Immunization 
Safety  Monitoring  (PRISM)  Network,  a 
collaborative  effort  between  multiple 
health  plans,  federal  and  state  health 
agencies,  for-profit  and  nonprofit  orga¬ 
nizations,  and  academic  institutions, 
was  established  to  address  these  poten¬ 
tial  limitations.  The  PRISM  Network 
linked  health  plan  and  Immunization  In¬ 
formation  Systems  (IIS)  data,  conducted 
continuous  active  surveillance  for  pre¬ 
specified  outcomes,  and  could  provide 
timely  information  on  unanticipated  po¬ 
tential  risks  if  needed.  PRISM  was  built 
on  a  distributed  research  network  in 
which  data  were  physically  held  and 
managed  by  each  data  owner.  Develop¬ 
ment  of  PRISM  benefited  from  previous 
experiences  with  the  VSD  and  a  recent 
study  in  which  GBS  among  adolescents 
who  received  the  meningococcal  conju¬ 
gate  vaccine  was  examined.^^  PRISM 
used  vaccine  exposure  and  claims- 
based  outcomes  from  5  large  health 
plans  that  covered  —26  million  people, 
together  with  vaccine-exposure  data 
from  IIS  in  8  states.  Registry-enhanced 
data  from  —14  million  persons  captured 
publicly  delivered  vaccine.  Data  from  the 
PRISM  network  started  to  be  available  in 
January  2010. 

The  Indian  Health  Service,  VA,  and  DoD 
adapted  their  data  systems  to  conduct 
RCA  in  as  harmonized  a  manner  as 
feasible  with  the  VSD  and  PRISM  Net¬ 
work  to  facilitate  data-sharing  and 
interpretation. 

Through  the  collaborative  efforts  of 
the  FDA  and  Centers  for  Medicare  and 
Medicaid  Services,  the  development  of 
methods  to  use  Medicare  data  for 
near-real-time  active  safety  surveil¬ 
lance  of  2009  H1N1  monovalent  influ¬ 
enza  vaccines  was  accelerated.  This 
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work  focused  on  monitoring  GBS.  Avail¬ 
ability  of  vaccinations  in  the  Medicare 
data  depends  on  providers  billing 
Medicare.  The  Centers  for  Medicare 
and  Medicaid  Services  created  new 
billing  codes  for  use  by  vaccine  provid¬ 
ers  to  distinguish  administration  of 
2009  MINI  monovalent  influenza  vac¬ 
cines  from  seasonal  influenza  vac- 
cines.26'27 

The  CDC  implemented  population- 
based  active  surveillance  at  10  sites 
aroundthe  country  that  participate  in  its 
Emerging  Infections  Program  (EIP)  to 
identify  cases  of  GBS.  The  catchment 
population  was  —45  million  people.  The 
primary  method  for  case  ascertainment 
was  through  a  network  of  neurologists 
and  hospitals.  GBS  cases  captured 
through  this  system  were  reviewed  by 
using  standardized  case  definitions.  GBS 
cases  identified  and  verified  through  this 
system  were  compared  with  the  ex¬ 
pected  number  of  GBS  cases  determined 
through  estimates  of  vaccine  coverage 
and  background  rates  of  GBS  available 
through  the  literature.  The  CDC  also  col¬ 
laborated  with  the  American  Academy  of 
Neurologyto  educate  neurologists  about 
reporting  to  the  VAERS  and  to  enhance 
GBS  case-finding. 

The  Vaccines  and  Medications  in  Preg¬ 
nancy  Surveillance  System  (VAMPSS) 
is  a  collaboration  of  the  Organization 
of  Teratology  Information  Specialists 
(OTIS),  Slone  Epidemiology  Center,  and 
the  American  Academy  of  Allergy, 
Asthma  and  Immunology.  The  VAMPSS 
received  funding  from  the  HHS  Office  of 
the  Assistant  Secretary  for  Prepared¬ 
ness  and  Response  and  its  Biomedical 
Advanced  Research  and  Development 
Authority.  The  VAMPSS  conducts  pro¬ 
spective  cohort  and  retrospective  case- 
control  studies  of  influenza  vaccines,  in¬ 
fluenza  antiviral  drugs,  and  natural 
influenza  exposure  and  maternal  and  fe¬ 
tal  outcomes.  The  OTIS  receives  calls 
from  pregnant  women  who  have  been 
exposed  to  a  variety  of  medical  and  envi¬ 


ronmental  substances  including  vac¬ 
cines.  The  OTIS  collects  information  on 
vaccine  exposure  during  pregnancy  and 
enrolls  subjects  in  cohort  and  case- 
control  studies.  As  of  May  2010,  the 
VAMPSS  was  still  in  the  process  of  study 
initiation  for  2009  MINI  monovalent  in¬ 
fluenza  vaccines. 

The  DoD’s  Naval  Health  Research  Center, 
through  its  Center  for  Deployment 
Health  Research  and  Birth  and  Infant 
Health  Registry,  initiated  both  a  prospec¬ 
tive  and  retrospective  analysis  among 
pregnant  women  who  received  2009 
H1N1  monovalent  influenza  vaccines.  The 
study  objective  is  to  describe  the  inci¬ 
dence  and  prevalence  of  adverse  repro¬ 
ductive  health  outcomes  among  female 
military  service  members  who  received 
the  vaccine  during  pregnancy.  As  of  May 
2010,  data  were  not  yet  available.  Final 
results  are  expected  within  1  year  of  the 
infants  reaching  their  first  birthday. 

The  CISA  Network  supported  2009  H1N1 
monovalent  influenza  vaccine-safety 
monitoring  by  collecting  medical  histo¬ 
ries  and  biological  samples  for  prespeci¬ 
fied  events  (including  many  listed  in  the 
Appendix)  reported  to  the  VAERS.  For  ex¬ 
ample,  some  vaccine  recipients  who  re¬ 
ported  GBS  to  the  VAERS  were  contacted 
by  CISA  Network  investigators  who  re¬ 
quested  serum  and  mucosal  swab  spec¬ 
imens  that  could  be  used  for  host  risk- 
factor  assessment  if  and  when  the  need 
arises. 

Table  2  provides  the  status  of  the  sys¬ 
tems  described  above  in  relation  to  2009 
H1N1  monovalent  influenza  vaccine- 
safety  monitoring  as  well  as  their 
strengths  and  limitations  for  signal 
strengthening  and  verification  and  as¬ 
sessment  of  association. 

Assessing  the  Evidence 

The  NVAC  H1N1  Vaccine  Safety  Risk  As¬ 
sessmentworking  Group  (VSRAWG)  was 
created  to  independently  assess  the 
safety  profile  of  2009  H1N1  monovalent 
influenza  vaccines  and  to  develop,  on  a 


timely  basis,  appropriate  information 
for  presentation  to,  and  deliberation  by, 
the  NVAC.  The  VSRAWG  included  a  broad 
range  of  expertise  important  for  assess¬ 
ing  vaccine  safety  and  included  repre¬ 
sentatives  from  5  federal  vaccine  advi¬ 
sory  committees  (Vaccines  and  Related 
Biological  Products  Advisory  Committee, 
Advisory  Committee  on  Immunization 
Practices,  NVAC,  National  Biodefense  Sci¬ 
ence  Board,  and  Defense  Health  Board) 
and  a  consumer  representative.  The  VS¬ 
RAWG  was  responsible  for  reviewing 
data  generated  by  the  systems  de¬ 
scribed  above  on  a  biweekly  basis  and 
more  frequently  as  necessary.  If  poten¬ 
tial  signals  were  identified,  the  VSRAWG 
was  charged  with  examining  possible 
associations  between  2009  H1N1  mon¬ 
ovalent  influenza  vaccines  and  the  ad¬ 
verse  event  of  interest.  The  NVAC  deliber¬ 
ated  on  the  information  presented  to  it 
by  the  VSRAWG,  forwarded  the  reports  of 
the  VSRAWG  (as  it  deemed  appropriate) 
to  the  HHS,  and  made  recommendations 
to  the  Assistant  Secretary  for  Health  on 
the  basis  of  the  NVAC  deliberations.  If  as¬ 
sociations  were  found,  the  VSRAWG 
would  advise  whether  the  association 
between  the  vaccine  and  the  adverse 
event  was  likely  caused  by  the  vaccine. 
The  VSRAWG  provided  monthly  reports 
that  summarized  its  findings  to  the 
NVAC.28  NVAC  reports  from  the  VSRAWG 
were  rapidly  shared  with  international 
health  authorities  and  made  publicly 
available. 

CONCLUSIONS 

The  US  government  made  a  number  of 
enhancements  to  existing  vaccine- 
safety  monitoring  systems  in  prepara¬ 
tion  for  the  2009  H1N1  monovalent  in¬ 
fluenza  vaccine  program.  Analyses  of 
data  are  ongoing  and  will  be  published 
once  complete.  Thus  far,  these  sys¬ 
tems  have  demonstrated  that  the  2009 
H1N1  monovalent  influenza  vaccines 
have  a  similar  safety  profile  to  that  of 
seasonal  influenza  vaccines,  which 
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TABLE  2  Signal  Strengthening  and  Verification  and  Assessment  of  Association  Programs  Strengths  and  Limitations 


Data  Source  and  Status  in  Relation  to 
2009  H1N1  Monovalent  Influenza 
Vaccination  Program 

VSD:  existing 


CMS  Medicare  data:  accelerated 
development  for  2009  H1N1 


PRISM:  newly  developed  for  2009  H1N1 


DMSS:  existing 


VA:  accelerated  development  for  2009 
H1N1 


CBS,  active  case-finding:  newly 
developed  for  2009  H1N1 


Indian  Health  Service:  newly 
developed  for  2009  H1N1 


VAMPSS:  newly  developed  for  2009 
H1N1 


Strengths 


Limitations 


Includes  people  from  all  life  phases  (eg,  children, 
adults,  pregnant  women,  etc) 

Rapid 

Experienced  (since  1991) 

Large  sample  (1.5  million  2009  H1N1  monovalent 
influenza  vaccine  doses  captured) 

Can  link  with  vaccine  registries  when  needed 
Chart-review  capability 
Predominately  aged  >65  y 
Rapid 

Very  large  (3  million  2009  HIM  monovalent 
influenza  vaccine  doses  captured,  primarily  for 
the  elderly) 

Includes  people  from  all  life  phases  (eg,  children, 
adults,  pregnant  women,  etc) 

Can  capture  vaccinations  not  in  medical  record 

Ascertainment  of  publicly  delivered  vaccine 

Rapid  once  system  fully  functional 
Very  large  (2.5  million  2009  H1N1  monovalent 
influenza  vaccine  doses  captured) 

Chart  review  possible 

Captures  the  DoD’s  mandatory  vaccination  program 
Clinical  data  from  the  electronic  health  record  on 
exposure  and  outcomes 

Large  (1.3  million  2009  MINI  monovalent  influenza 
vaccine  doses  captured) 

Rapid 

Experienced 

Chart  reviews  possible 

Includes  elderly  and  federal  employees  (other  than 
those  of  the  DoD) 

Rapid 

Large  (1.2  million  2009  MINI  monovalent  influenza 
vaccine  doses  captured) 

Chart  reviews  possible 

Large  sample  size  (catchment  population  of  45 
million  people) 

Timely 

Chart  reviews 

Includes  people  from  all  life  phases  (eg,  children, 
adults,  pregnant  women,  etc) 

Includes  minority  population 
Moderate-sized  population  (321  000  2009  HI  N1 
monovalent  influenza  vaccine  doses  captured) 
Vaccination  and  adverse  events  collected  via  unified 
system 

Chart  reviews  possible 

Captures  exposures  in  a  variety  of  settings 

Captures  outcomes  of  exposures  both  prospectively 
and  retrospectively 


Does  not  capture  all  vaccinations  received  outside  the 
MCOs 

Delays  in  receiving  data  on  health  encounters  and 
hospitalizations  outside  of  MCO  hospitals 


Database  developed  for  administrative  claims  data 
Elderly  not  initially  part  of  2009  HIM  vaccine  priority 
group 

Capturing  vaccination  data  depends  on  providers  billing 
Medicare 

Incomplete  capture  of  vaccinations 

Limited  ability  to  distinguish  between  specific  2009  H1N1 
vaccine  types  (ie,  live  vs  inactivated) 

Publicly  delivered  vaccine  will  be  limited  to  states  that 
use  the  Immunization  Information  System 
Data-sharing  for  vaccine  safety  untested 
Database  developed  for  administrative  claims  data 

Data  started  to  become  available  in  January  2010 
Healthy  population 
Limited  ages  represented 

Limited  experience  in  real-time  surveillance 

Database  developed  for  traditional  epidemiologic  studies 
focused  on  military-unique  exposures 

focused  primarily  on  veteran-unique  exposures 

Vaccine  database  not  well  tested 
Limited  experience  in  vaccine-safety  studies 

Imprecision  in  making  comparisons  to  background  rates 
developed  from  other  sources 
Imprecision  in  vaccinated  population  estimated  from 
survey  data 

Limited  experience  with  vaccine-safety  studies 


Long  lag  time  until  most  outcomes  of  interest  may  occur 

Potential  for  selection  and  recall  bias 

Moderate  sample  size  may  cause  small  or  moderate 
risks  of  very  rare  adverse  events  to  go  undetected 
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have  an  excellent  safety  profile.  An 
evaluation  is  planned  for  all  programs 
and  systems  developed  to  monitor 
2009  MINI  monovalent  influenza  vac¬ 
cine  safety.  These  programs  may 
prove  useful  for  future  mass  and  rou¬ 
tine  vaccination  programs.  In  addition, 
these  programs  may  lead  to  a  sustain- 
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APPENDIX  Prespecified  Outcomes  and  Definitions  Used  for  RCA 


Influenza 
Vaccine  Type 

Outcome 

ICD-9  CM  Code 

Primary  Postvaccination 
Follow-up  Window 
(Secondary  Window),  d 

All 

GBS 

357.0 

1-42 

All 

Myelitis,  acute  disseminated  encephalomyelitis,  and  other 
encephalitis 

323.5,  323.51,  323.52,  323.6,  323.61,  323.62, 

323.63,  323.8,  323.81,  323.82,  323.9,  and  341.2 

1-42  (1-21) 

All 

Bell's  palsy 

351.0 

1-60  (1-42) 

All 

Anaphylaxis 

995.0  and  999.4 

0-2 

All 

Other  demyelinating  disease  (multiple  sclerosis, 

demyelinating  disease  of  central  nervous  system,  optic 
neuritis,  chronic  inflammatory  demyelinating 
polyneuropathy) 

340,  341 .0,  341 .8,  341 .9,  377.30,  377.31 ,  377.32, 
377.34,  377.39,  and  357.81 

1-42 

All 

Disorders  of  the  peripheral  nervous  system  and  neuropathies 
(peripheral  autonomic  neuropathy,  mononeuritis, 
peripheral  neuropathy,  polyneuropathy  due  to  drugs  or 
other  toxic  agents,  critical  illness  polyneuropathy,  other 
inflammatory  and  toxic  neuropathy) 

337.0,  337.9,  354.1-354.9,  355.0-355.9,  356.4, 
356.8,  357.6,  357.7,  357.82,  357.89,  and  357.9 

1-42 

All 

Seizures  (epilepsy,  convulsions) 

345.00-345.91,  780.3,  780.31,  and  780.39 

0-7  (0-14) 

All 

Other  cranial  nerve  disorders 

350.1-350.9,  351.1,  351.8,  351.9,  and 

352.0-352.9 

1-42 

All 

Ataxia 

334.3 

1-42 

All 

Angioneurotic  edema,  allergic  reaction,  urticaria 

995.1,  995.3,  708.0,  708.1,  and  708.9 

1-2 

All 

Spontaneous  abortion,  missed  abortion 

632  and  634.0-634.9 

1-14 

All 

Stillborn 

V27.1,  V27.3,  V27.4,  V27.7,  V32,  V35,  and  V36 

1-14 

All 

Preeclampsia,  eclampsia 

642.4,  642.5,  642.6,  and  642.7 

1-14 

All 

Hemorrhagic  stroke 

430,  431,  and  432.0-432.9 

1-42 

All 

Ischemic  stroke 

433.01,  433.11,  433.21,  433.31,  433.81, 433.91, 
and  434.0-434.9 

1-42 

All 

Immune  thrombocytopenia 

287.31,287.4,  and  287.5 

1-42  (7-28) 

Live 

attenuated 

Myocarditis,  pericarditis 

420.90,  420.91,  422.0,  422.90,  422.91,  and  422.99 

1-42 

Live 

attenuated 

Asthma/wheezing 

493.0,  493.1, 493.9,  786.07,  and  519.11 

1-14 

Live 

attenuated 

Asthma/wheezing  /bronchiolitis 

466.1,  466.11,  466.19,  493.0-493.9,  786.07, 

786.09,  and  519.1 

1-14  days  (1-42) 

ICD-9  indicates  International  Classilication  of  Diseases,  Ninth  Revision. 
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